Information processing apparatus and non-transitory computer readable medium

ABSTRACT

An information processing apparatus includes a processor configured to: acquire, from results of character recognition performed on a target image including character strings, attribute information indicating an attribute to which a key character string and a value character string belong, the key character string as a character string specified beforehand as a key and the value character string as a character string indicating a value corresponding to the key character string; acquire by using the attribute information the key character string corresponding to the value character string extracted from the results of the character recognition; and output the key character string and the value character string corresponding to the key character string.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2020-208692 filed Dec. 16, 2020.

BACKGROUND (i) Technical Field

The present disclosure relates to an information processing apparatusand a non-transitory computer readable medium.

(ii) Related Art

Japanese Patent No. 5621169 discloses a form recognition apparatus thatreads a read-target character string from a form group including avariety of layouts and determines an attribute of the character stringwithout any definition of a form. The form recognition apparatusincludes a character string detection unit, character string recognitionunit, item-name likelihood calculation unit, item value likelihoodcalculation unit, placement likelihood calculation unit, item-name toitem-value relation evaluation value calculation unit, and item-name toitem-value relation determination unit. The character string detectionunit detects a character string region from a form image. The characterstring recognition unit recognizes individual characters in thecharacter string region. The item name likelihood calculation unitcalculates an item name likelihood representing a probability of acharacter string being an item name in the form image. The item valuelikelihood calculation unit calculates an item value likelihoodrepresenting a probability of a character string in the form imagematching grammatical and notation rules of a word or a character stringregistered in a representation dictionary. The placement likelihoodcalculation unit calculates a placement likelihood. The placementlikelihood indicates whether a placement of a pair of character stringsin the form image is appropriate in an item-name to item-valuerelationship in terms of a character string frame of the characterstring pair or a character string rectangle. The item-name to item-valuerelation evaluation value calculation unit calculates an evaluationvalue that represents a likelihood of the character string pair as anitem name to item value in accordance with the item name likelihood,item value likelihood, and placement likelihood. The item-name toitem-value relation determination unit determines association of anitem-name to item-value relation in the form image in response to theevaluation value output by the item-name to item value-relationevaluation value calculation unit.

Japanese Patent No. 6616269 discloses a technique that causes a computerof a form processing apparatus to function as elements described below.The elements include an image reading unit, character string recognitionunit, same-row character string group information acquisition unit,specific character string determination unit, specific imagedetermination unit, and content item acquisition unit. The image readingunit acquires a form image by causing an image reading apparatus toreach the form. The character string recognition unit recognizes acharacter string by performing a character recognition operation on theform image acquired by the image reading unit. The same-row characterstring group information acquisition unit acquires same-row characterstring group information about a character string group at the same rowfrom among character strings recognized by the character stringrecognition unit. The specific character string determination unitdetermines whether a predetermined character string is included in eachpiece of the same-row character sting group information acquired by thesame-row character string group information acquisition unit. Thespecific image determination unit determines whether a predeterminedspecific image is present in the vicinity of the same-row characterstring group information that is determined as including the specificcharacter string by the specific character string determination unit. Ifthe specific image determination unit has determined that the specificimage is present, the content item acquisition unit acquires as aspecific content item described in the form an item character stringincluded in the same-row character string group information in thevicinity of the specific image.

Techniques of extracting a character string included in an image byperforming an optical character recognition (OCR) operation on the mageread from a document are available. To extract a character string froman image through the OCR operation, key value extraction may beperformed to extract a character string having a value (hereinafterreferred to as a value character string) corresponding to a characterstring specified beforehand as a key (hereinafter referred to as a keycharacter string).

In the key value extraction via the OCR operation, the value characterstring may be extracted from results of the OCR operation. However, akey character string may be difficult to extract because of possibleerratic recognition or because the key character string corresponding tothe value character string may not be included in a document. In such acase, the key character string and the value character string aredifficult to output.

SUMMARY

Aspects of non-limiting embodiments of the present disclosure relate toproviding an information processing apparatus and a non-transitorycomputer readable medium outputting a key character string and a valuecharacter string even when the key character string is difficult toextract via the OCR operation or the key character string correspondingto the value character string is not included in a document.

Aspects of certain non-limiting embodiments of the present disclosureaddress the above advantages and/or other advantages not describedabove. However, aspects of the non-limiting embodiments are not requiredto address the advantages described above, and aspects of thenon-limiting embodiments of the present disclosure may not addressadvantages described above.

According to an aspect of the present disclosure, there is provided aninformation processing apparatus including a processor configured to:acquire, from results of character recognition performed on a targetimage including character strings, attribute information indicating anattribute to which a key character string and a value character stringbelong, the key character string as a character string specifiedbeforehand as a key and the value character string as a character stringindicating a value corresponding to the key character string; acquire byusing the attribute information the key character string correspondingto the value character string extracted from the results of thecharacter recognition; and output the key character string and the valuecharacter string corresponding to the key character string.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present disclosure will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram illustrating a hardware configuration exampleof an information processing apparatus of each exemplary embodiment;

FIG. 2 is a block diagram illustrating a functional configurationexample of the information processing apparatus of a first exemplaryembodiment;

FIG. 3 illustrates a target image example from which a character stringis extracted in accordance with each of the exemplary embodiments;

FIG. 4 illustrates a recognition result example of each of the exemplaryembodiments;

FIG. 5 illustrates an example of a positional relationship database ofeach of the exemplary embodiments;

FIG. 6 illustrates an example of a confirmation correction screen thatis used to specify a key character string and a value character stringin accordance with each of the exemplary embodiments;

FIG. 7 is a flowchart illustrating an example of a extraction process toextract a character string in accordance with the first exemplaryembodiment;

FIG. 8 is a block diagram illustrating a functional configurationexample of the information processing apparatus of a second exemplaryembodiment; and

FIG. 9 is a flowchart illustrating an extraction process example toextract a character string in accordance with each of the exemplaryembodiments.

DETAILED DESCRIPTION First Exemplary Embodiment

First exemplary embodiment of the disclosure is described below withreference to the drawings.

A configuration of an information processing apparatus 10 is describedwith reference to FIG. 1. FIG. 1 is a block diagram illustrating ahardware configuration example of the information processing apparatus10 of a first exemplary embodiment. For example, the informationprocessing apparatus 10 of the first exemplary embodiment includes butis not limited to a terminal, such as a personal computer, or a server.The information processing apparatus 10 may be built in anotherapparatus, such as an image forming apparatus.

Referring to FIG. 1, the configuration of the information processingapparatus 10 includes a central processing unit (CPU) 11, read-onlymemory (ROM) 12, random-access memory (RAM) 13, storage 14, input unit15, monitor 16, and communication interface (I/F) 17. The CPU 11, ROM12, RAM 13, storage 14, input unit 15, monitor 16, and communication I/F17 are interconnected to each other via a bus 18. The CPU 11 is anexample of processor.

The CPU 11 controls the whole of the information processing apparatus10. The ROM 12 stores a variety of programs, including an informationprocessing program, and data used in the first exemplary embodiment. TheRAM 13 is a memory used as a work area when the variety of programs areexecuted. The CPU 11 extracts a character string by expanding theprogram from the ROM 12 onto the RAM 13 and executing the program. Forexample, the storage 14 is a hard disk drive (HDD), solid-state drive(SSD), or flash memory. The information processing program may also bestored on the storage 14. The input unit 15 includes a mouse andkeyboard receiving input characters. For example, the monitor 16displays an extracted character. The communication I/F 17 transmits orreceives data.

Referring to FIG. 2, the functional configuration of the informationprocessing apparatus 10 is described. FIG. 2 is a block diagramillustrating a functional configuration example of the informationprocessing apparatus 10 of the first exemplary embodiment.

Referring to FIG. 2, the information processing apparatus 10 includes anacquisition unit 21, recognition unit 22, determination unit 23,extraction unit 24, confirmation correction unit 25, output unit 26,memory 27 and learning unit 28A. When the CPU 11 executes theinformation processing program, the information processing apparatus 10functions as the acquisition unit 21, recognition unit 22, determinationunit 23, extraction unit 24, confirmation correction unit 25, outputunit 26, memory 27 and learning unit 28A.

The acquisition unit 21 acquires an image 31 serving as a target(hereinafter referred to as a target image 31) from which a characterstring is extracted. Referring to FIG. 3, the target image 31 of thefirst exemplary embodiment is the image of a document that includes butis not limited to an entry item, a character string written by a userfor the entry item, and an object 32, such as an imprint. The targetimage 31 may be an image of a form or a slip delineated by ruled lines,a mechanically printed receipt, or any other image as long as itincludes a character string. The object 32 may now be positioned at alocation that is predetermined depending on the type of the document.

Via the optical character recognition (OCR) operation, the recognitionunit 22 acquires from the target image 31 a character string and theobject 32 included in the document and positions (coordinates) of thecharacter string and the object 32 in the target image 31 and outputsthese pieces of information as recognition results 33.

Referring to FIG. 4, for example, the recognition results 33 of thefirst exemplary embodiment include the character string and the object32 acquired from the target image 31 and the attribute, type, andposition (coordinates) of each of the character string and the object32. For example, the attribute of each character string in therecognition results 33 refers to information, such as “date” and“address” indicating the attribute to which the character string belongs(hereinafter referred to as “attribute information”). The type refersthe type of a character string representing a specified key (hereinafterreferred to as a “key character string”) or the type of a characterstring representing a value corresponding to the key character string(hereinafter referred to as a “value character string”).

According to the first exemplary embodiment, the recognition unit 22recognizes a character string included in the target image 31.Alternatively, the recognition unit 22 may analyzes the target image 31to identify the type of the document and output recognition results. Forexample, in the analysis, a specific character string and a position ofa ruled line are recognized and the recognition results are comparedwith pre-stored feature of documents to identify the type.Alternatively, an identifier identifying a document in the target image31 may be recognized to identify the type of the document. By causingthe type of the document to be identified, the key character stringincluded in each document may be identified. Specifically, therecognition unit 22 may identify a character string to be extracted byidentifying the type of the document.

The determination unit 23 determines the attribute information of eachcharacter string in the recognition results 33 and the type of thecharacter string in the recognition results 33. The determination unit23 outputs the attribute determined on each character string and thetype of the character string to the recognition results 33. Thedetermination unit 23 is a learning model that has learned to determinethe attribute and type of the character string. For example, thedetermination unit 23 may be recurrent neural network (RNN) and supportvector machine (SVM). The determination unit 23 learns beforehand thecharacter string, attribute of the character string, and type of thecharacter string, determines the attribute and type of the characterstring using the character string output by the recognition unit 22, andoutputs determination results to the recognition results 33.

The determination unit 23 of the first exemplary embodiment describedabove is the learning model that has learned to determine the attributeand type of the character string. The disclosure is not limited to thismethod. For example, the attribute and type of the character string maybe determined by using a character string pre-stored on the memory 27 tobe discussed below. In this case, the determination unit 23 derives adegree of similarity between the character string output from therecognition unit 22 and the character string stored on the memory 27.The determination unit 23 may determine as the attribute and type of thecharacter string the attribute and type of a character string having thehighest degree of similarity from among the character strings stored onthe memory 27. The degree of similarity may be derived from Levenshteindistance. The Levenshtein distance is determined by counting the numberof character replacements, character additions, and character deletionswhen any character string is changed into another character string.

The extraction unit 24 extracts the key character string correspondingto the value character string from the recognition results 33 byacquiring the position of the value character string from therecognition results 33 and by using the acquired position. Specifically,the extraction unit 24 extracts as a key character string a characterstring having the same attribute as the value character string and beingin the vicinity of the value character string. For example, thecharacter string in the vicinity refers to a character string positionedwithin a predetermined distance from the position of the value characterstring or a character string being closest in distance to the positionof the value character string. For example, if “Taro Fuji” in FIG. 3 isacquired as the value character string, the extraction unit 24 extractsas the key character string “applicant name” having the same attributeas the value character string.

According to the first exemplary embodiment, the character string in thevicinity of the value character string is extracted as the key characterstring. The disclosure is not limited to this method. For example, acharacter string in a predetermined direction may be extracted as a keycharacter string. For example, a character string positioned in apredetermined direction from the value character string, for example,positioned on the left-hand side of the value character string, may beextracted as the key character string. Referring to FIG. 5, the memory27 described below may store a positional relationship database(hereinafter referred to as a “positional relationship DB”) 34 thatassociates the attribute, key name, and positional relationship. Theattribute is an attribute of a character string, the key name is a nameof a key character string described in the document in the target image31, and the positional relationship indicates a direction of thecorresponding value character string from each key character string. Thepositional relationship DB 34 is an example of relation information.

Referring to FIG. 5, positional relationship “K-Right-V” in theapplication date indicates that, in the target image 31, the valuecharacter string “month, day, year” is positioned on the right-hand sideof the key character string “application date”. In other words, thepositional relationship indicates that the key character string“application date” is positioned on the left-hand side of the valuecharacter string “month, day, year”.

The extraction unit 24 acquires the positional relationship related tothe value character string from the positional relationship DB 34 byusing the attribute of the value character string extracted from therecognition results 33. The extraction unit 24 may acquire as the keycharacter string a character string positioned in a direction oppositeto the direction indicated by the acquired positional relationship withrespect to the position of the value character string.

The confirmation correction unit 25 displays the character string, andthe attribute, type, and position of the character string extracted fromthe target image 31 and receives a correction to the attribute, type,and position of the character string. For example, referring to FIG. 6,the confirmation correction unit 25 displays a confirmation correctionscreen 40. The confirmation correction screen 40 includes an extractioncharacter string display region 41 and a target image display region 42.The confirmation correction unit 25 displays as an extracted characterstring a character string extracted by the extraction unit 24 in theextraction character string display region 41 and high-lights theposition of a character string corresponding to the displayed characterin the target image 31 in the target image display region 42.

After the extracted character string displayed in the extractioncharacter string display region 41 is selected, the confirmationcorrection screen 40 accepts a correction to a character string that isextracted by specifying the position corresponding to the extractedcharacter string in the target image display region 42 and a correctionto the position of the character string in the target image 31. Forexample, after the “application date” is selected in the extractioncharacter string display region 41, the user may specify a region wherethe “application date” is described. A correction to the key characterstring and the position of the key character string is ready to beaccepted. In this case, a color column corresponding to the “applicationdate” in the extraction character string display region 41 and theregion of the application date in the target image display region 42 arehigh-lighted with the same color.

The output unit 26 outputs the key character string and value characterstring extracted from the target image 31.

The memory 27 stores, in association with each other, the target image31, the character string extracted from the target image 31, and theposition of the character string in the target image 31. The memory 27also stores the positional relationship DB 34, the target image 31having undergone extraction, and the character string in the targetimage 31 having undergone the extraction.

The learning unit 28A learns the determination unit 23 serving as alearning model. The learning unit 28A causes the determination unit 23to learn with the target image 31 and character string being as inputdata and with the attribute and type of the character string being asteacher data.

In the above discussion, the determination unit 23 of the firstexemplary embodiment learns with the character string being the inputdata and determines the attribute and type of the character string. Thedisclosure is not limited to this method. The position of the characterstring in the target image 31 may be treated as the input data. Forexample, using as the input data the position of the character stringincluded in the recognition results 33 of the target image 31, thedetermination unit 23 may learn and determine the attribute and type ofthe character string.

The determination unit 23 may also determine the attribute and type ofthe character string by learning a relationship between the position ofthe character string in the target image 31 and the position of theobject 32 in the target image 31. For example, if two terms “applicationaddress” and “applicant address” having the same attribute representedby the character string “address” in a document may indicate differentaddresses. The term “application address” does not have a key characterstring in the target image 31. However, positions of an entry item andthe object 32 in the document are predetermined on a per document basis.The attribute and type of the character string may be identified basedon the positional relationship between the object 32 and the characterstring in the target image 31. Specifically, the determination unit 23may determine the attribute and type of the character string by learningas the input data the relationship between the position of the object 32in the target image 31 and the position of the character string in thetarget image 31.

The determination unit 23 may determine the attribute and type of thecharacter string by learning the positional relationship of thecharacter strings as the input data or by learning as the input data thepositional relationship included in the positional relationship DB 34and the positions of the character strings. If the key character stringcorresponding to the value character string is not included in thetarget image 31, the determination unit 23 may set the determinedattribute as the key character string or may determine the key characterstring corresponding to the value character string.

The process of the information processing apparatus 10 of the firstexemplary embodiment is described with reference to FIG. 7. FIG. 7 is aflowchart illustrating an extraction process example to extract acharacter string in accordance with the first exemplary embodiment. TheCPU 11 reads the information processing program from the ROM 12 or thestorage 14 and executes the information processing program. Theinformation processing program in FIG. 7 is thus executed. Theinformation processing program in FIG. 7 is executed when the userinputs the target image 31 and an instruction to perform the extractionprocess.

In step S101, the CPU 11 acquires the target image 31 input by the user.

In step S102, the CPU 11 performs the OCR operation on the acquiredtarget image 31 and outputs a character string and a position of thecharacter string as the recognition results 33.

In step S103, the CPU 11 determines the attribute and type of thecharacter string by using the recognition results 33 and outputs thedetermination results into the recognition results 33.

In step S104, the CPU 11 searches for and extracts the value characterstring on the recognition results 33.

In step S105, the CPU 11 identifies and extracts the key characterstring corresponding to the value character string by using the positionand attribute of the extracted value character string.

In step S106, the CPU 11 determines whether the key character string hasbeen extracted. If the key character string has been extracted (yes pathin step S106), the CPU 11 proceeds to step S108. If the key characterstring is not extracted (no path in step S106), the CPU 11 proceeds tostep S107.

In step S107, the CPU 11 sets the attribute of the value characterstring to be a key character string.

In step S108, the CPU 11 outputs as the extraction results the keycharacter string and the value character string in association with eachother.

In step S109, the CPU 11 determines whether another value characterstring is present. If the other value character string is present (yespath in step S109), the CPU 11 returns to step S104. If no other valuecharacter string is present (no path in step S109), the CPU 11 proceedsto step S110.

In step S110, the CPU 11 displays the confirmation correction screen andreceives a user correction to the attribute, type, and position of thecharacter string.

In step S111, the CPU 11 stores the target image 31, the attribute,type, and position of the character string in association with eachother.

In step S112, the CPU 11 outputs the key character string and valuecharacter string in accordance with the extraction results.

As described above, according to the first exemplary embodiment, the keycharacter string corresponding to the value character string isextracted by using the attribute of the character strings. The key andvalue extraction is thus performed. If a key character string is notextracted through the OCR operation or a key character stringcorresponding to the value character string is not included in thedocument, a key character string as a corresponding character string andthe value character string are output.

Second Exemplary Embodiment

According to the first exemplary embodiment, if the key character stringcorresponding to the value character string is extracted from therecognition results 33, the key character string and the value characterstring are output in association with each other. According to a secondexemplary embodiment, if the key character string corresponding to thevalue character string is not extracted from the recognition results 33,an image corresponding to the key character string is detected from thetarget image 31 and the key character string and the value characterstring are output in association with each other.

The hardware configuration of the information processing apparatus 10 ofthe second exemplary embodiment (see FIG. 1) is identical to thehardware configuration of the first exemplary embodiment. For example,the target image 31 (see FIG. 3), the recognition results 33 (see FIG.4), and the positional relationship DB 34 (see FIG. 5) remain unchangedand the discussion thereof is omitted herein. The confirmationcorrection screen 40 (see FIG. 6) also remains unchanged from the firstexemplary embodiment and the discussion thereof is also omitted herein.

Referring to FIG. 8, the functional configuration of the informationprocessing apparatus 10 is described below. FIG. 8 is a block diagramillustrating a functional configuration example of the informationprocessing apparatus 10 of the second exemplary embodiment. In FIG. 8,functions of the information processing apparatus 10 identical to thoseof the information processing apparatus 10 in FIG. 2 are designated withthe same reference numerals and the discussion thereof is omittedherein.

Referring to FIG. 8, the information processing apparatus 10 includes anacquisition unit 21, recognition unit 22, determination unit 23,extraction unit 24, confirmation correction unit 25, output unit 26,memory 27, learning unit 28B, and detection unit 29. By executing theinformation processing program, the CPU 11 functions as the acquisitionunit 21, recognition unit 22, determination unit 23, extraction unit 24,confirmation correction unit 25, output unit 26, memory 27, learningunit 28B, and detection unit 29.

The learning unit 28B performs a learning process on the determinationunit 23 as a learning model and the detection unit 29. The learning unit28B causes the determination unit 23 to learn with the target image 31and the character string being as the input data and with the attributeand type of the character string being as the teacher data. The learningunit 28B causes the detection unit 29 to learn with the target image 31and the position of the character string being as the input data andwith the key character string corresponding to the value characterstring being as the teacher data.

Via an object detection process, the detection unit 29 detects a keycharacter string in the vicinity of the value character string specifiedin the target image 31. Specifically, the detection unit 29 is alearning model, such as convolution neural network (CNN) and you onlylook once (YOLO), which has performed machine learning to detect acharacter string in the vicinity of a specified character string. Thedetection unit 29 detects from the target image 31 an image of a keycharacter string positioned in the value character string, acquires akey character string by identifying the detected image, and outputs thekey character string to the determination unit 23.

The detection unit 29 of the second exemplary embodiment is a learningmodel using machine learning. The detection unit 29 detects from thetarget image 31 an image of the key character string positioned in thevicinity of the position of the value character string. The disclosureis not limited to this method. The image of the key character stringpositioned in the vicinity of the position of the value character stringmay be detected via a pattern matching operation. For example, eachimage corresponding to the key character strings may be storedbeforehand and the image of a key character string may be detected viathe pattern matching operation, such as shape detection or templatematching. Also, the detection unit 29 may detect the key characterstring from the target image 31 by using the image corresponding to thekey character string and identify the key character string positioned inthe vicinity of the value character string.

The process of the information processing apparatus 10 of the secondexemplary embodiment is described with reference to FIG. 9. FIG. 9 is aflowchart illustrating a extraction process example to extract acharacter string in accordance with the second exemplary embodiment. TheCPU 11 reads the information processing program from the ROM 12 or thestorage 14 and executes the information processing program. Theinformation processing program in FIG. 9 is thus executed. Theinformation processing program in FIG. 9 is executed when the userinputs the target image 31 and an instruction to perform the extractionprocess. In FIG. 9, steps identical to those in the extraction processin FIG. 7 are designated with the same reference numerals and thediscussion thereof is omitted herein.

In step S113, the CPU 11 determines whether a key character string isextracted. If the key character string is extracted (yes path in stepS113), the CPU 11 proceeds to step S108. If no key character string isextracted (no path in step S113), the CPU 11 proceeds to step S114.

In step S114, the CPU 11 performs a detection operation to detect a keycharacter string in the target image 31 by using the position of thevalue character string and outputs the key character string as detectionresults. In the detection operation, the image of the key characterstring corresponding to the value character string is detected from thetarget image 31 and the key character string is acquired.

In step S115, the CPU 11 determines in response to detection results theattribute and type of the character string and outputs the attribute andtype of the character string into the detection results.

In step S116, the CPU 11 determines whether the detected key characterstring corresponds to the value character string. If the detected keycharacter string corresponds to the value character string (yes path instep S116), the CPU 11 proceeds to step S108. On the other hand, if thedetected key character string does not correspond to the value characterstring (no path in step S116), the CPU 11 proceeds to step S117.

In step S117, the CPU 11 sets the attribute of the value characterstring to be the key character string.

As described above, the key and value extraction is performed bydetecting the key character string corresponding to the value characterstring via the detection operation. Even when the key character stringis not extracted via the OCR operation, a key character string as acorresponding character string and the value character string areoutput.

According to the second exemplary embodiment, the key character stringis detected via the detection operation. Alternatively, the valuecharacter string may be detected.

According to the exemplary embodiments, a correction to the attribute,type, and position of the character string is received on theconfirmation correction screen 40. The disclosure is not limited to thismethod. A correction to the extracted character string may be received.If a correction to the extracted character strings is received, thecorrection may be accepted with all the extracted character stringsuniformly displayed. Alternatively, a degree of certainty indicatingcertainty of the character string may be derived. If the degree ofcertainty is lower than a predetermined threshold, for example, only acharacter string having certainty lower than the predetermined thresholdmay be displayed.

According to the exemplary embodiments, a correction to the position ofthe character string acquired from the recognition results 33 isaccepted on the confirmation correction screen 40. The disclosure is notlimited to this method. The position of the character string in thetarget image 31 may be specified.

According to the exemplary embodiments, a correction to the position ofthe character string is received in the confirmation correction. Thedisclosure is not limited to this method. When the target image 31 isinput to the information processing apparatus 10, the designate ofposition of the character string may be received beforehand.Alternatively, the target image 31 stored on the memory 27 may bedisplayed at any time point after the output of the character string,and then a correction to the position of the character string may bereceived.

According to the exemplary embodiments, a correction to the characterstring is received on the confirmation correction screen 40. Thedisclosure is not limited to this method. The extracted character stringmay be corrected by using the character string stored on the memory 27or the character string extracted in the past. Multiple value characterstrings may be stored in association with each other on the memory 27and a value character string associated with an extracted valuecharacter string may be searched for and displayed. For example, thevalue character string of “name” and value character string of “address”extracted in the past may be stored in association with each other onthe memory 27. If the value character string of “name” is extracted, theextraction unit 24 may acquire from the memory 27 the value characterstring of “address” corresponding to the value character string of“name” and may display a substitute candidate for correction.

The extracted character string may be corrected by using a learningmodel based on machine learning. For example, the character stringextracted from the target image 31 having undergone the extractionoperation and the character string corrected in the past are stored onthe memory 27. A correction unit (not illustrated) learns the targetimage 31 stored on the memory 27, the character string in the targetimage 31 and the character string corrected in the past. The correctionunit may display a correction candidate of the extracted characterstring to correct the character string.

In the embodiments above, the term “processor” refers to hardware in abroad sense. Examples of the processor include general processors (e.g.,CPU: Central Processing Unit) and dedicated processors (e.g., GPU:Graphics Processing Unit, ASIC: Application Specific Integrated Circuit,FPGA: Field Programmable Gate Array, and programmable logic device).

In the embodiments above, the term “processor” is broad enough toencompass one processor or plural processors in collaboration which arelocated physically apart from each other but may work cooperatively. Theorder of operations of the processor is not limited to one described inthe embodiments above, and may be changed.

According to the exemplary embodiments, the information processingprogram is installed on the storage. The disclosure is not limited tothis method. The information processing program of the exemplaryembodiments may be distributed in a recorded form on a computer readablerecording medium, such as an optical disc, like a compact-disc read-onlymemory (CD-ROM) or digital versatile disc ROM (DVD-ROM). Alternatively,the information processing program may be distributed in a recorded formon a semiconductor memory, such as a universal serial bus (USB) or amemory card. The information processing program of the exemplaryembodiments may be acquired from an external apparatus via acommunication network, such as the communication I/F 17.

The foregoing description of the exemplary embodiments of the presentdisclosure has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit thedisclosure to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the disclosure and its practical applications, therebyenabling others skilled in the art to understand the disclosure forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of thedisclosure be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing apparatus comprising aprocessor configured to: acquire, from results of character recognitionperformed on a target image including character strings, attributeinformation indicating an attribute to which a key character string anda value character string belong, the key character string as a characterstring specified beforehand as a key and the value character string as acharacter string indicating a value corresponding to the key characterstring; acquire by using the attribute information the key characterstring corresponding to the value character string extracted from theresults of the character recognition; and output the key characterstring and the value character string corresponding to the key characterstring.
 2. The information processing apparatus according to claim 1,wherein the processor comprises a determination model that has learnedto determine an attribute to which a character string belongs, andwherein the processor is configured to, by using the determinationmodel, determine an attribute of a character string included in thetarget image.
 3. The information processing apparatus according to claim2, wherein the processor is configured to: cause the determination modelto learn beforehand a character string and an attribute of the learnedcharacter string; and determine an attribute of a character stringextracted from the results of the character recognition.
 4. Theinformation processing apparatus according to claim 2, wherein theprocessor is configured to: acquire position information indicating aposition of the character string in the target image; cause thedetermination model to learn beforehand a position of a character stringin an image and an attribute of the character string whose position hasbeen learned; and determine, by using a position of a character stringextracted from the results of the character recognition, an attribute ofthe character string extracted from the results of the characterrecognition.
 5. The information processing apparatus according to claim4, wherein the processor is configured to: cause the determination modelto learn a positional relationship between an object at a predeterminedposition in an image and a position of a character string in the image;and determine an attribute in accordance with the positionalrelationship between a position of an object in the target image and aposition of the character string in the target image.
 6. The informationprocessing apparatus according to claim 1, wherein the processorcomprises a detection model that has learned to detect a characterstring included in an image, and wherein the processor is configured to,if the key character string has not been detected from the results ofthe character recognition on the target image, extract the key characterstring by using the detection model.
 7. The information processingapparatus according to claim 2, wherein the processor comprises adetection model that has learned to detect a character string includedin an image, and wherein the processor is configured to, if the keycharacter string has not been detected from the results of the characterrecognition on the target image, extract the key character string byusing the detection model.
 8. The information processing apparatusaccording to claim 3, wherein the processor comprises a detection modelthat has learned to detect a character string included in an image, andwherein the processor is configured to, if the key character string hasnot been detected from the results of the character recognition on thetarget image, extract the key character string by using the detectionmodel.
 9. The information processing apparatus according to claim 4,wherein the processor comprises a detection model that has learned todetect a character string included in an image, and wherein theprocessor is configured to, if the key character string has not beendetected from the results of the character recognition on the targetimage, extract the key character string by using the detection model.10. The information processing apparatus according to claim 5, whereinthe processor comprises a detection model that has learned to detect acharacter string included in an image, and wherein the processor isconfigured to, if the key character string has not been detected fromthe results of the character recognition on the target image, extractthe key character string by using the detection model.
 11. Theinformation processing apparatus according to claim 6, wherein theprocessor is configured to: acquire positional relationship informationindicating a positional relationship between the key character stringand the value character string; and detect by using the detection modelthe key character string in accordance with the positional relationshipinformation and a position of the key character string extracted fromthe results of the character recognition.
 12. The information processingapparatus according to claim 7, wherein the processor is configured to:acquire positional relationship information indicating a positionalrelationship between the key character string and the value characterstring; and detect by using the detection model the key character stringin accordance with the positional relationship information and aposition of the key character string extracted from the results of thecharacter recognition.
 13. The information processing apparatusaccording to claim 8, wherein the processor is configured to: acquirepositional relationship information indicating a positional relationshipbetween the key character string and the value character string; anddetect by using the detection model the key character string inaccordance with the positional relationship information and a positionof the key character string extracted from the results of the characterrecognition.
 14. The information processing apparatus according to claim9, wherein the processor is configured to: acquire positionalrelationship information indicating a positional relationship betweenthe key character string and the value character string; and detect byusing the detection model the key character string in accordance withthe positional relationship information and a position of the keycharacter string extracted from the results of the characterrecognition.
 15. The information processing apparatus according to claim10, wherein the processor is configured to: acquire positionalrelationship information indicating a positional relationship betweenthe key character string and the value character string; and detect byusing the detection model the key character string in accordance withthe positional relationship information and a position of the keycharacter string extracted from the results of the characterrecognition.
 16. The information processing apparatus according to claim1, wherein the processor is configured to, by using a pre-stored valuecharacter string or a value character string corrected in past, correctthe value character string extracted from the results of the characterrecognition and output the corrected value character string.
 17. Theinformation processing apparatus according to claim 16, wherein theprocessor is configured to, by using a correction model, correct thevalue character string extracted from the results of the characterrecognition, the correction model having learned the pre-stored valuecharacter string or the value character string corrected in past. 18.The information processing apparatus according to claim 1, wherein theprocessor is configured to, if the key character string corresponding tothe value character string is not extracted, set the attributeinformation in the key character string and output the key characterstring with the attribute information and the value character stringcorresponding to the key character string.
 19. The informationprocessing apparatus according to claim 1, wherein the processor isconfigured to: acquire relation information that associates theattribute information with positional relationship informationindicating a positional relationship between the key character stringand the value character string; and acquire the key character stringcorresponding to the value character string that has been extracted inaccordance with the position of the value character string extractedfrom the results of the character recognition and the positionalrelationship information associated with the attribute informationindicating the attribute of the value character string in the relationinformation.
 20. A non-transitory computer readable medium storing aprogram causing a computer to execute a process for processinginformation, the process comprising: acquiring, from results ofcharacter recognition performed on a target image including characterstrings, attribute information indicating an attribute to which a keycharacter string and a value character string belong, the key characterstring as a character string specified beforehand as a key and the valuecharacter string as a character string indicating a value correspondingto the key character string; acquiring by using the attributeinformation the key character string corresponding to the valuecharacter string extracted from the results of the characterrecognition; and outputting the key character string and the valuecharacter string corresponding to the key character string.